■;  Naval  Aerospace  Medical.  Research  Laboratory  L 


: 


■  '.A  ■  V 

NAMRL-jt  1  3  36 

.  In.  V  ■ 


ec  ember  1 987' 


DEVELOPMENT  AND 
AN  AUTOMATED 


AND  MULTIPLE^iliiiTlO 


>  m-iYW'vi 


•  •"'••I*  .%•'§§ 

■  ■  -  ■ 


'•••  •■:;•  Hi-jw 

.  .  ‘  j,,m, 


UNCLASSIFIED 


SECURITY  CLASSIFICATION  OF  THIS  PAGE 


REPORT  DOCUMENTATION  PAGE 

la.  REPORT  SECURITY  CLASSIFICATION 

lb  RESTRICTIVE  MARKINGS 

2a.  SECURITY  CLASSIFICATION  AUTHORITY 

Unclassified 

3.  DISTRIBUTION/ AVAILABILITY ,QF  REPORT 

Approved  for  public  release; 

2b.  DECLASSIFICATION /DOWNGRADING  SCHEDULE 

distribution  unlimited. 

4.  PERFORMING  ORGANIZATION  REPORT  NUMBER(S) 

S.  MONITORING  ORGANIZATION  REPORT  NUMBER(S) 

NAMRL  -  1336 

6a.  NAME  OF  PERFORMING  ORGANIZATION  1 6b.  OFFICE  SYMBOL  I  7a.  NAME  OF  MONITORING  ORGANIZATION 


Naval  Aerospace  Medical 
Research  Laboratory 


6c.  ADDRESS  (City,  State,  and  ZIP  Code) 

NAS,  Pensacola,  FL  32508-5700 


(If  applicable)  Naval  Medical  Research  and  Development 

03  Command 


7b.  ADDRESS  {City,  State,  and  ZIP  Code) 

NMC,  NCR,  3ethesda,  MD  20814-5044 


8a.  NAME  OF  FUNDING /SPONSORING 
ORGANIZATION 


8b.  OFFICE  SYMBOL  9.  PROCUREMENT  INSTRUMENT  IDENTIFICATION  NUMBER 
(If  applicable) 


8c.  ADDRESS  (City,  State,  and  ZIP  Code) 


10.  SOURCE  OF  FUNDING  NUMBERS 


PROGRAM  j  PROJECT  I  TASK 

ELEMENT  NO.  I  NO.  NO 


WORK  UNIT 
ACCESSION  NO. 


63706N  I  M0096001  I  001  I  0151 


11.  TITLE  (Include  Security  Claudication) 

(u)  Development  and  Evaluation  of  an  Automated  Series  of  Single  and  Multiple  Dichotic 
Listening  and  Psychomotor  Tasks 


12.  PERSONAL  AUTHOR(S) 
G.  R.  Griffin 


13a.  TYPE  OF  REPORT 

13b.  TIME  COVERED 

14.  DATE  OF  REPORT  (Year,  Month,  Day) 

15.  PAGE  COUNT 

Interim 

FROM  1987  TO  1987 

87-12 

16 

16.  SUPPLEMENTARY  NOTATION 

17. 

COSATI  CODES  | 

FIELD 

GROUP 

SUB-GROUP  1 

18.  SUBJECT  TERMS  (Continue  on  reverse  if  neceuary  and  identify  by  block  number) 
Pilot  Selection,  Psychomotor,  Dichotic  Listening,  Multi¬ 
task  performance 


19.  ABSTRACT  ( Continue  on  reverse  if  neceuary  and  identify  by  block  number) 

Attrition  in  undergraduate  naval  aviation  training  represents  a  costly  problem.  An 
average  of  25%  of  student  naval  aviators  fails  to  complete  training.  This  study  reports 
an  effort  to  develop  automated  single-  and  multiple-dichotic  listening  and  psychomotor 
tasks,  which  have  the  potential  to  reduce  aviator  attrition  through  improved  selection  and 
may  also  be  useful  in  initial  pipeline  classification. 

Statistical  analysis  of  Study  I,  comparing  a  forward  and  backward  series  of  automated 
dichotic  listening  (DLT)  and  psychomotor  (PMT)  tasks,  indicated  that  a  backward-direction 
orientation  associated  with  the  psychomotor  tests  resulted  in  increased  difficulty  for  all 
PMT  measures  and  two  of  three  multitask  DLT  measures.  The  correlational  estimates  of 
tesc-retest  reliability  for  the  multitask  DLT  and  PMT  measures  were  adequate  for  both 
series  of  automated  tasks  but  slightly  higher  (r  *  .80  DLT,  r  “  .90  PMT)  for  the 


21.  ABSTRACT  SECURITY  CLASSIFICATION 
Unc lassif ied 


20.  DISTRIBUTION  /  AVAILA8ILITY  OF  ABSTRACT 
□  l&CLASSIFIED/UNLIMITED  □  SAME  AS  PPT.  □  DTlC  USERS 


22a.  NAME  OF  RESPONSIBLE  INDIVIDUAL  22b.  TELEPHONE  (Include  Area  Code)  j  22c.  OFFICE  SYMBOL 

J.  0.  HOUGHTON,  GAPT  MCUSN  (904)  452-3286  00 


DO  FORM  1473,  84  MAR  83  APR  edition  may  be  used  until  exhausted.  SECURITY  CLASSIFICATION  OF  THIS  PAGE 

All  other  editions  are  obsolete 
i 


UNCLASSIFIED 


Security  classification  ot  this  page:  UNCLASSIFIED 


19.  ABSTRACT  (Continued) 

backward  series.  There  were  significant  correlations  between  the  DLT  and  PMT  tasks,  for 
each  series  of  single-  and  multitask  measures.  The  relation  between  such  seemingly 
different  tasks  is  difficult  to  understand  since  the  DLT  is  an  auditory  cognitive 
processing  task,  and  the  PMT  is  an  eye,  hand,  foot  coordination  task.  However,  the 
significant  correlations  were  both  smaller  and  fewer  in  number  for  the  backward  series  of 
automated  tests. 

Study  II  was  a  correlational  evaluation  between  the  new  automated  multitask  measures  and 
old  nonautomated  tasks  with  demonstrated  validity  for  the  prediction  of  primary  flight 
performance.  The  correlations  between  corresponding  tasks  of  the  new  automated  and 
old  nonautomated  tasks  averaged  .60  for  the  DLT  measures  and  ,66  for  the  PMT  tasks. 

The  results  of  Study  III  indicated  that  certain  automated  DLT  and  PMT  measures  were 
significantly  related  to  primary  flight  grades  (PFG)  in  Navy  flight  training.  For  the 
backward  series  of  tasks,  all  DLT  and  PMT  measures  were  significantly  correlated  with  PFG* 
However,  only  two  DLT  and  two  PMT  measures  of  the  forward  series  were  significantly 
related  to  FFG.  No  significant  correlations  were  found  between  the  automated  DLT  and  PMT 
tests  and  the  pass/fail  criterion.  The  absence  of  a  suitable  number  of  flight  failure 
attritions  was  discussed  as  a  possible  reason  for  this  result.  A  regression  analysis  for 
the  backward  series  of  test  measures  and  primary  flight  training  criteria  indicated  that  a 
psychomotor  stick  and  rudder  measure  and  the  FAR  selection  test  were  significantly  related 
to  PFG  (R  =  .53,  £  (2,85)  ■*  16.56,  £  <  .0001).  There  were  no  significant  correlations 
between  the  automated  DLT  and  PMT  measures  and  prior  flight  hours. 

These  results  indicate  that  a  series  of  automated  DLT  and  PMT  tasks  are  suitable 
replications  of  an  older  version  of  nonautomated  tasks.  The  advantages  of  the  automated 
tasks  are  that  they  require  less  administrative  support  and  provide  automatic  scoring  ot 
performance.  The  backward  series  of  automated  tasks,  which  was  correlated  more  strongly 
with  criterion  performance,  should  be  administered  to  a  large  sample  of  student  naval 
aviators  to  determine  if  the  tests  can  account  for  additional  variance  in  the  prediction 
of  flight  training  performance  beyond  chat  of  current  selection  tests. 


ii 


Security  classification  of  this  page:  UNCLASSIFIED 
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THE  PROBLEM 

Attrition  in  undergraduate  naval  aviation  training  is  a  costly  problem. 
An  average  of  '*5%  of  student  naval  tviators  fails  to  complete  training. 

This  study  reports  an  effort  to  develop  automated  single-  and  multiple- 
dichotic  listening  and  psychomotor  tasks,  which  have  the  potential  to  reduce 
aviator  attrition  through  improved  selection  and  may  also  be  useful  in 
initial  pipeline  classification. 

FINDINGS 

Statistical  analysis  in  Study  I,  comparing  automated  Dichotic  Listening 
(DLT)  and  Psychomotor  (PMT)  tasks,  indicated  that  a  backward-direction 
orientation  associated  with  the  stick  movement  of  one  of  the  psychomotor 
tasks  resulted  in  increased  difficulty  for  all  PMT  measures  and  two  of  three 
DLT  measures  performed  under  multitask  conditions.  The  correlational 
estimates  of  test-retest  reliability  for  the  multitask  DLT  and  PMT  measure* 
were  adequate  for  both  series  of  automated  tasks  but  slightly  higher  (r 
=  .80  DLT,  v_  *  .90  PMT)  for  the  backward  series.  There  were  significant 
correlations  between  the  DLT  and  PMT  tasks  for  each  series  of  single-  and 
multitask  measures.  The  relation  between  such  seemingly  different  tasks  is 
difficult  to  understand  since  the  DLT  is  an  auditory  cognitive  processing 
task,  and  the  PMT  is  an  eye,  hand,  and  foot  coordination  task.  However,  the 
significant  correlations  were  both  smaller  and  fewer  in  number  for  the 
backward  series  of  automated  tests. 

Study  II  was  a  correlational  evaluation  between  the  new  automated 
multitask  measures  and  the  old  nonautomated  tasks  with  demonstrated 
validity  tor  tne  prediction  of  primary  flight  performance.  The  correlations 
between  corresponding  tasks  of  the  new  automated  and  old  nonautomated  tasks 
averaged  .60  for  the  DLT  and  .66  for  the  PMT. 

The  results  of  Study  III  indicated  that  certain  of  the  automated  DLT 
and  PMT  measures  were  significantly  related  to  primary  flight  grades  (PFG) 
in  Navy  flight  training.  For  the  backward  series  of  tasks,  all  DLT  and  PMT 
measures  were  significantly  correlated  with  PFG.  However,  only  two  DLT  and 
two  PMT  measures  of  the  forward  series  were  significantly  related  to  PFG. 

No  significant  correlations  were  found  between  the  automated  DLT  and  PMT 
tests  and  the  pass/fail  criterion.  The  absence  of  a  suitable  number  of 
flight  failure  attritions  was  discussed  as  a  possible  reason  for  this 
result.  A  regression  analysis  for  the  backward  series  of  test  measures  and 
primary  flight  training  criteria  indicated  that  a  psychomotor  stick-and- 
rudder  measure  ;\nd  the  FAR  selection  test  were  significantly  related  to  PFG 
(_R  m  * o 3 ,  (2,85)  =  16.56,  £  <  .0001).  There  were  no  significant 

correlations  between  the  automated  DLT  and  PMT  measures  and  prior  flight 
hours. 

RECOMMENDATION 


These  research  results  indicate  that  a  series  of  automated  DLT  and  PMT 
tasks,  which  require  less  administrative  support  and  provide  automatic 
scoring  of  performance,  are  suitable  replications  of  an  older  version  of 
nonautomated  tasks.  The  backward  serias  of  automated  tasks,  which  was 
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correlated  more  strongly  with  criterion  performance,  should  be  administered 
to  a  large  sample  of  student  naval  aviatorn  to  determine  if  the  tests  can 
account  for  additional  variance  in  the  prediction  of  flight  training 
performance  beyond  that  of  current  selection  tests. 
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INTRODUCTION 


A  system  of  automated  single  and  multiple  tacks  has  been  developed  at 
the  Naval  Aerospace  Medical  Research  Laboratory  (NAMRL)  to  support  test 
batteries  for  performance-based  biomedical  selection  and  classification  of 
naval  aviators  (1).  Certain  or  these  automated  tasks  were  based  on 
previous  resear:h  demonstrating  the  potential  of  nonautomated  multiple 
tasks  to  improve  human  performance  in  simulated  flight  (2)  and  to  predict 
naval  aviator  performance  in  training  (3)  and  fleet  environments  (4). 

A  previously  reported  traditional  dual-performance  experiment  (2) 
indicated  that  the  performance  of  subjects  on  the  psychomotor  portion  of  a 
combined  psychomotor  (Systems  Research  Laboratories,  Inc.,  Psychomotor 
Task)  and  dichotic  listening  task  (DLT)  was  significantly  improved  when  a 
vocal,  as  opposed  to  manual,  response  mode  was  employed  for  dichotic 
listening.  A  subsequent  validation  study  (3)  based  on  student  naval 
aviator  performance  on  the  single  and'  combined  tasks  indicated  that  the 
single  tasks  were  valid  predictors  of  pass/fail  and  primary  training  flight 
grades  (£  <  .05).  Importantly,  the  best  predictor  of  pass/fail  (j>  <  .01) 
was  the  DLT  performed  under  multitask  conditions.  Alternately,  the  best 
predictor  of  primary  flight  grades  was  a  single-task  stick-and-rudder 
psychomotor  (PMT)  measure.  Multiple  regression  results  indicated  that  the 
multitask  DLT,  the  Flight  Aptitude  Rating  (FAR)  selection  test,  and 
multitask  PMT  psychomotor  measures  accounted  for  significant  variance  in 
the  prediction  of  the  pass/ fail  criterion.  A  separate  regression  analysis 
demonstrated  that  the  multitask  DLT  and  PMT  measures  accounted  for 
significant  variance  beyond  that  of  the  Naval  and  Marine  aviation  selection 
test  battery  (3). 

An  additional  regression  analysis  was  performed  on  the  data  referred 
to  above  (3)  using  primary  flight  grades  as  the  criterion.  The  ability  to 
predict  flight  grades  is  an  important  consideration  in  the  event  of  a  Navy 
management  requirement  to  assign,  prior  to  primary  t  Ining,  student  naval 
aviators  into  fixed-  or  rotary-wing  pipelines.  In  thru.  regression 
analysis,  a  single- task  psychomotor  stick-and-rudder  measure  entered  the 
regression  equation  first  -  .51)  followed  by  the  AQT  ( R  -  .64).  Each  of 
these  measures  contributed  significant  variance  to  the  final  regression 
equation  (j?  (2,43)  “  14.89,  <  .01).  This  analysis  suggests  that 

psychomotor  and  selection  test  measures  may  be  useful  in  initial  aviator 
classification.  In  addition  to  the  above,  a  preliminary  automated  version 
of  the  PMT  and  DLT  was  used  successfully  to  predict  the  air  combat 
maneuvering  performance  of  an  F-4  squadron  undergoing  a  fleet  readiness 
evaluation  at  NAS  Oceana  (4). 

The  DLT  and  PMT  tasks  have  been  automated  at  NAMRL  on  an  Apple  lie 
computer  as  a  series  of  single-  and  multitask  tests.  The  task  series 
and  cumulative  testing  times  are  depicted  in  Table  1.  The  purpose  of  the 
present  study  was  to  report  descriptive  statistics  and  reliability 
estimates  of  two  versions  of  the  automated  task  series  (Study  I),  present 
correlational  statistics  indicating  the  relation  between  the  original 
nonautomated  and  newly  automated  tasks  (Study  II),  and  estimate  the 
relation  of  the  automated  series  of  tasks  to  flight  training  performance 
(Study  III). 
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TABLE  1.  Automated  Psychoraotor  and  Diehotic  Listening  Task  Menu. 

Automated  task  Cumulative 

presentation  order  Description  teat  time  (min) 


L.  Single- task  psychomotor,  stick  only  09 

2.  Single-task  diehotic  listening  32 

3.  Initial  multitask*  37 

4.  Single- task  psychomotor,  stick  &  rudder  50 

5.  Second  multi  task®  55 

6.  Third  multitask*3  60 


tasks 

1 

and 

2 

combined 

tasks 

4 

and 

2 

combined 

STUDY  L.  COMPARISON  OF  TWO  AUTOMATED  SERIES  OF  SINGLE  AMD  MULTIPLE  TASKS 

The  objective  of  this  evaluation  was  to  report  and  compare  the 
descriptive  statistics  and  correlate onal  reliability  estimates  of  two 
series  of  single  and  multiple  tasks.  One  series,  labeled  a  forward  test 
version,  incorporated  psychomotor  stick-and-rudder  movements  that 
corresponded  to  respective  cursor  movement  on  a  CRT.  That  is,  when  the 
stick  or  rudder  was  moved  to  the  right  or  left,  the  CRT  cursor  moved  to  the 
right  or  left.  A  second  series,  or  backward  test  version,  incorporated 
stick-and-rudder  movements  that  were  opposite  CRT  cursor  movement. 

Subjects 

One  hundred  student  naval  aviators  performed  on  the  automated  diehotic 
listening  and  psychomotor  tasks.  Fifty  subjects  (Group  I)  nerformed  on  the 
forward  series  of  tasks,  and  fifty  subjects  (Group  II)  performed  on  the 
backward  series  of  single  and  multiple  tasks. 

Apparatus  and  Procedure 


All  subjects  performed  each  series  of  automated  tasks  in  identical 
order  (see  Table  1).  Subjects  were  volunteers  about  to  enter  navy  flight 
training  during  the  fall  of  1986.  A  photograph  of  the  test  apparatus  is  in 
Figure  1. 

Psychomotor  Single  Task  (PMT).  Subjects  were  required  to  maintain 
first  one  and  then  two  computer-generated  cursors  on  fixed  targets 
presented  on  a  CRT  using  a  Measurement  Systems,  Inc.,  joystick  end  Locally 
produced  rudder  pedals  patterned  after  those  of  a  Systems  Research 
Laboratory  Psychoraotor  test  device.  The  subjects  manipulated  the  joystick 
using  their  right  hand  to  control  the  stick  X,Y-movement  cursor  and 
manipulated  rudder  pedals  using  their  feet  to  control  a  second  Z-movement 
cursor.  Single- task  PMT  performance  consisted  of  two  3-rniti  sessions  of  a 
stick-only  PMT  task  followed  by  three  3-min  sessions  of  a  stick-and-rudder 
combined  task.  Each  of  the  single- task  PMT  6-  and  9-rain  test  sessions  were 
preceded  by  3  min  of  p’  actice.  Individual  sessions  were  separated  by  rest 
periods  of  20  «.  All  instructions  were  presented  visually  on  a  CRT. 
Psychomotor  test  scores  were  computer-calculated  pixel  error  over  the  6- 
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and  9-min  test  periods,  respectively.  The  error  scores  were  totaled  for 
the  X,Y  *cick  cursors  and  X,Y,Z  atick-and-rudder  cursors  and  represent  sums 
of  the  linear  deviations  of  the  cursors  from  an  ideal  non-error  position 
for  each  of  three  movement  axes  (X,Y,  or  Z)  of  the  visual  display  (see  Fig. 
1). 


Figure  1.  Automated  dichotic  listening  and  psychomotor  task  apparatus. 


Dlchotic  Listening  Single  Task  (DLT).  The  DLT  was  patterned  after 
that  of  Gopher  (5),  and  subsequently  modified  and  then  automated  at  NAMRL 
(6).  The  DLT  is  an  auditorially  presented  series  of  letter-digit  string 
sets  for  24  trials.  Two  Jameco  JE  520-AP  Voice  Synthesizers  were  used  to 
present  the  DLT  letter-digit  strings  over  binaural  headphones  to  each 
subject  at  a  listening  level  of  72  dB/Leq  (re:20f/pa).  Subjects  were 
instructed  to  direct  attention  to  one  ear  while  ignoring  the  other  and  to 
correctly  report,  using  a  keypad,  the  digits  (0-9)  presented  to  the 
a  trended- to-ear  in  the  sequence  of  their  occurrence.  Subject  responses 
were  made  on  a  keypad  placed  immediately  in  front  and  slightly  left  of 
center,  for  left-hand  use.  Test  instructions  were  presented  visually  on 
the  CRT,  and  the  test  was  preceded  by  six  auditorially  presented  practice 
trials  with  immediate  visual  performance  feedback  indicating  the  letters 
and  digits  presented  and  the  subjects'  keypad  responses.  Finally,  subjects 
completed  three  multiple-choice  questions  to  make  certain  that  they 
understood  the  concept  of  the  DLT.  An  example  of  a  DLT  trial  is  written  in 
Table  2.  The  single- task  DL1  performance  measure  was  the  number  of  correct 
responses  per  24  trials;  216  correct  responses  were  possible. 
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TABLE  2.  LLT  Trial  Visual  Example 


PART  I  Left  Ear  R8NSMY2GB7FL6RL5 

"Right"  (Vocal  Channel  'attend'  Command) 

Right  Ear  YL3SR4FZ9XFQFN1L 


PART  II  Left  Ear  B  F  4  3  7  9 

"LEFT  (Vocal  Channel  'attend'  Command) 
Right  Ear  G  L  1  5  6  2 


Multitask  PMT/DLT.  Subjects  performed  the  multitask  PMT/DLT  with 
their  right  hand  on  the  joystick  and  feet  on  the  rudder  pedals  (PMT)  and 
used  tneir  left  hand  for  keypad  responses  to  the  auditorially  presented 
DLT.  In  the  multitask  conditions,  subjects  performed  three  sessions  of  the 
DLT  and  PMT  simultaneously  (a  12- trial  DLT  and  a  4.5-min  PMT).  During  the 
initial  multitask  condition,  subjects  performed  the  DLT  and  stick-only  PMT. 
During  the  latter  two  multitask  conditions,  subjects  performed  the  DLT  and 
the  stick-and-rudder  PMT.  Performance  measures  for  the  PMT  and  DLT  in  the 
multitask  conditions  were  identical  to  single- task  performance  except  for 
different  lengths  of  PMT  testing  and  the  presentation  of  12,  rather  than 
24,  DLT  trials  during  multitask  performance.  One  hundred  and  eight  correct 
responses  were  possible  for  the  reduced  set  of  DLT  trials. 

Results 


Descriptive  statistics  for  the  forward  and  backward  series  of  single 
and  multiple  tasks  are  provided  in  Table  3.  An  analysis  of  variance 
indicated  a  significant  difference  between  the  two  (_F  (1,98)  ■  33.77,  £ 

<  .01).  Further,  separate  follow-on  tests  (Duncan,  Tukey,  F-test)  (7) 
indicated  that  all  measures  of  the  two  series,  except  the  DLT  single  task 
and  the  third  DLT  multitask  measures  reflect  significantly  different 
subject  performance.  The  backward  series  of  single  and  multiple  tasks  was 
more  difficult.  In  two  of  the  three  multitask  conditions,  the  increased 
difficulty  associated  with  the  backward  PMT  not  only  resulted  in  larger 
error  scores  for  all  PMT  measures  but  more  DLT  errors  as  well. 

The  correlation  matrices  in  Tables  4A  and  4B  identify  statistically 
significant  correlations  between  the  DLT  and  PMT  forward  and  backward 
series  of  single-  and  multitask  measures,  and  present  estimates  of  multitask 
measure  reliabilities  as  well.  The  correlation  coefficients  indicate 
acceptable  reliabilities  for  the  multiple  tasks.  For  the  DLT,  the  relation 
between  the  last  two  multitask  DLT  trials  (measures  3  and  4  of  Tables  4A 
and  4B)  is  jr  »  .79  for  the  forward  series  and  £  =>  .80  for  the  backward 
series  of  tasks.  For  the  PMT,  the  correlation  between  the  last  two 
multitask  trials  (measures  8  and  9  of  Tables  4A  and  4B)  is  r  *  .82  for  the 
forward  series  and  jr  =»  .90  for  the  backward  series  of  tasks. 

A  disconcerting  aspect  f  both  the  forward  and  backward  series  of 
automated  tasks  is  that  there  are  significant  correlations  between  the  DLT 
and  tasks  in  both  single-  and  multitask  performance  conditions. 

Thi3  finding  has  been  reported  previously  (3).  However,  it  is  difficult  to 
understand  how  the  DLT  and  PMT  tasks  are  related  when  they  appear  so  very 
different — one  being  an  eye,  hand,  and  foot  coordination  task  and  the  other 
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an  auditory  listening  task.  One  might  expect  significant  correlations 
between  the  tasks  performed  in  combination  since  performance  on  one  task 
affects  performance  on  the  other.  For  example,  measures  2  and  8  of  Tables 
4A  and  4B  (initial  multitask  DLT  and  PMT)  are  significantly  correlated,  (r^ 
■  -.69  for  the  forward  series  and  j:  «  -.43  for  the  backward  series  of 
tasks).  Results  ?lso  indicate  that  even  the  single  DLT  and  PMT  tasks  are 
significantly  related.  For  example,  the  correlation  between  the  single¬ 
task  DLT  (measure  1)  and  the  PMT  single- task  s tick-and-rudder  (measure  7) 
is  -.34  for  the  forward  series  and  -.29  for  the  backward  series  of  tasks. 


TABLE  3.  Descriptive  Statistics  for  a  Forward  and  a  Backward  Series 
(Parentheses)  of  Single-  and  Multitask  Measures.3 

Measure  ””  —— 

DLT  correct  responses 


PMT  pixel  errors 

Mean 

SD 

Min 

Max 

1. 

Single- task  DLT 

206.76 

7.83 

169 

215 

(205.40) 

(6.54) 

(189) 

(215) 

2i 

Initial  multitask  DLT 

103.20 

5.61 

77 

108 

(99.80) 

(5.88) 

(82) 

(108) 

3. 

Second  multitask  DLT 

100.54 

7.99 

58 

108 

(97.14) 

(9.07) 

(72) 

(107) 

4. 

Third  multitask  DLT 

101.56 

8.06 

58 

108 

(99.00) 

(8.30) 

(74) 

(107) 

5. 

Single- task  PMT 

6971.04 

3322.38 

4462 

28115 

stick  only 

(13255.12) 

(7655.64) 

(4907) 

(41838) 

6. 

Initial  multitask  PMT 

2807.28 

1290.16 

1459 

9129 

stick  only 

(6166.90) 

(5052.80) 

(1360) 

(26207) 

7. 

Single- task  PMT 

26266.36 

14680.13 

13360 

115424 

stick  &  rudder 

(46041. 00)(23664. 11) 

(18399) 

(107625) 

8. 

Second  multitask  PMT 

7145.28 

4398.95 

2382 

29224 

stick  &  rudder 

(16833. 72)(15005. 29) 

(4935) 

(93195) 

9. 

Third  multitask  PMT 

6941.56 

3298.56 

2518 

16295 

stick  &  rudder 

(15495. 42)(13872. 92) 

(3720) 

(84015) 

Importantly,  the  correlations  between  the  DLT  and  PMT  measures  in  the 
backward  series  of  tasks  are  smaller  and  therefore  more  desirable. 

(A  goal  of  test  battery  development  is  to  identify  predictive  unrelated  tests 
so  ..hat  each  may  contribute  variance  to  a  suitable  criterion.)  The 
correlations  between  measure  1  (single- task  DLT)  and  the  single-  and  multitask 
PMT  measures  5  through  9  indicate  only  two  significant  correlations  for  the 
backward  series  compared  to  four  for  the  forward  series  of  tasks. 


.50  Forward  +  50  Backward) 
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In  summary,  the  correlational  results  indicate  slightly  higher  (but 
not  significantly  different)  reliabilities  for  the  backward  series  of 
tasks.  More  importantly,  less  relation  exists  between  the  DLT  and  PMT 
measures  in  the  backward  series  of  tasks.  Finally,  there  is  increased 
difficulty  for  both  the  DLT  and  PMT  measures  in  the  backward  series  of 
tasks . 


TABLE  4A.  Pearson  Product  Moment  Correlation  Matrix  for  a 
Forward  Series  of  DLT  and  PMT  Measures  (n  =  50) . 


17* 

— 

2. 

.78** 

-- -- 

3. 

.63** 

.56** 

— 

4. 

.66** 

.57** 

.79** 

— 

5. 

-.13 

-.02 

-.09 

-.13 

---- 

6  • 

-.68** 

-.68** 

-.39** 

-.34* 

.18 

— — 

7. 

-.34* 

-.27* 

-.09 

-.09 

.10 

.46** 

— 

8. 

-.72** 

-.69** 

-.52** 

-.53** 

.06 

.84** 

.54**  - 

9. 

-.49** 

-.47** 

-.54** 

-.52** 

.05 

.65** 

.41**  .82**  - 

TABLE  4B. 

son  Product  Moment  Correlation  Matrix  for  a 

Ut- Kward  Series  of  DLT  and  PMT  Measures  (_n  =  50). 

F 

•  ■“**“*“ 

2. 

.49** 

-  -  -  - 

3. 

.31* 

,65**  - 

4. 

.32* 

.48**  .80**  - 

5. 

.04 

-.19  -.22  -.13 

---- 

6. 

.  12 

-.23  -.23  -.08 

.65** 

---- 

7. 

-.29* 

-.58**  -.51**  -.45** 

.47** 

.47** 

---- 

8. 

-.34** 

-.43**  -.36**  -.26 

.31* 

.35* 

.65**  - 

9. 

a- 

-.26 

-.40**  -.31*  -.20 

.29* 

.47** 

.67**  .90**  - 

s  See  Table  3  tor  a  description  of  numbered  variables 
*p  <  .05  (two-tailed) 

**jj  <  .01  (two-taiLed) 


STUDY  II.  RELATION  OF  AUTOMATED  DLT  AND  PMT  TASKS  TO  OLD  NONAUTOMATED 
PROTOTYPES 

The  objective  of  Study  II  was  to  evaluate  the  correlation  between  the 
new  automated  DLT  and  PMT  tasks  and  the  old  tape  recorded/presented  DLT  and 
Systems  Research  Laboratory,  Inc.,  (SRL)  psychomotor  tasks.  In  essence, 
this  represents  a  validation  process  to  verify  that  the  automated  tasks 
measure  performance  similar  to  that  of  the  nonautomafed  tasks.  The 
advantages  of  an  automated  series  of  tasks  are  that  two  test  administrators 
would  not  be  needed  to  collect  and  analyze  performance  data,  and  the 
scoring  of  performance  could  be  made  automatic. 
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Subjects 


Sixty  student  naval  aviators  performed  on  the  forward  and  backward 
series  of  automated  tasks  and  the  old  nonautomated  DLT  and  PMT.  All 
subjects  were  volunteers  awaiting  entry  into  the  primary  portion  of  the 
Navy  flight  training  program.  Not  all  subjects  completed  all  portions  of 
the  various  tasks  uue  to  scheduling  and  equipment  problems. 

Apparatus  and  Procedure 

Subjects  initially  performed  on  the  new  automated  forward  series  of 
DLT  and  PMT  tasks.  Four  days  later,  subjects  performed  on  the  backward 
series  of  automated  tasks  and  immediately  thereafter  completed  one  5-min 
practice  and  one  5-rain  test  trial  on  the  old  (backward-oriented  cursor 
configuration)  nonautomated  multitask  DLT  and  SRL  PMT.  The  apparatus  for 
the  new  automated  DLT  and  PMT  tasks  was  identical  to  that  described  in 
Study  I.  The  apparatus  for  the  old  multitask  DLT  and  PMT  has  been 
described  in  the  introduction  section  and  in  previous  reports  (2,3). 

Resul ts 


Table  5  presents  the  correlations  between  the  nine  measures  of  the  new 
automated  forward  series  with  the  new  automated  backward  series  taken  4  days 
later.  As  expected,  there  are  significant  correlations  between  the  two  test 
series.  Naturally,  one  would  expect  high  correlations  between  the  DLT  tasks 
since  they  are  identical  in  the  two  series.  In  one  sense,  the  correlations 
amount  to  a  test/ re  test  reliability  measure  over  an  intervening  period  of  4 
day3.  The  correlations  between  the  PMT  scores  are  also  highly  significant. 
These  results  indicate  that  those  subjects  who  performed  well  on  the  initial 
forward  series  of  tasks  continued  to  perform  relatively  well  on  the  more 
difficult  backward  task  series  and  vice  versa.  In  summary,  although  Study  1 
results  indicated  a  significantly  higher  degree  of  difficulty  associated  with 
the  backward  series  of  tasks,  there  is  still  a  significant  and  strong 
correlation  between  subject  performance  on  the  two  series  of  tasks. 


TABLE  5.  Pearson  Product  Moment  Correlations  between  the  Forward 


and  Backward  Series  of  DLT  and  PMT  Tasks  (n  =  56). 


Measure  r 


1.  Single- task  DLT  .69** 

2.  Initial  multitask  DLT  .82** 

3.  Second  multitask  DLT  ,82** 

4.  Third  multitask  DLT  ,82** 

5.  Single  PMT  stick  .28* 

6.  Initial  multitask  PMT  stick  .50** 

7.  Single  PMT  stick  &  rudder  .75** 

8.  Second  multitask  PMT  stick  &  rudder  .82** 

9.  Third  multitask  PMT  stick  &  rudder3  .74** 


“  n  =  50 

*£  <  .05  (two-tailed) 
**j?  <  .01  (two-tailed) 
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The  correlation  statistics  of  Table  6  indicate  a  significant  relation 
between  toe  new  automated  multitask  DLT  and  PMT  measures  and  those 
corresponding  to  the  old  apparatus.  The  correlations  for  the  PMT  measures 
are  generally  stronger  than  those  of  the  DLT. 


TABLE  6.  Pearson  Product  Moment  Correlations  between  the  New 
Forward  and  Backward  Automated  Task  Series  and  Old 
Nona utoma ted  DLT  and  PMT. 


Forward  automated 
multitask  measures 
( a  =  40 ) 

Nona utoma ted 
multitask  DLT 

Nonautoma ted 
multitask  PMT 

Initial  multitask  DLT 

.60** 

Second  multitask  DLT 

.66** 

Third  multitask  DLT 

.58** 

InitiaL  multitask  PMT  stick 

.62** 

Single  PMT  stick  &  rudder 

.73** 

Second  multitask  PMT  stick  & 

rudder 

.66** 

Third  multitask  PMT  stick  & 

ruddera 

.53** 

Backward  automated 

Nona utoma ted 

Nonautoma ted 

multitask  measures 

multitask  DI.T 

multitask  PMT 

(n  =  37) 


Initial  multitask  DLT 

.49** 

Second  multitask  DLT 

.61** 

Third  multitask  DLT 

.63** 

Initial  multitask  PMT  Stick 

.69** 

Single  PMT  stick  &  rudder 

.66** 

Second  multitask  PMT  stick  A 

rudder 

.70** 

Third  multitask  PMT  stick  & 

Cl  AC 

rudder 

.69** 

“  n  =  35 

**r>  <  .01  (two-tailed) 


STUDY  HI.  RELATION  OF  THE  AUTOMATS?  FORWARD  AND  BACKWARD  SERIES  OF  PMT  AND 
DLT  TASKS  TO  TRAINING  PERFORMANCE  CRITERIA 

The  purpose  of  Study  III  was  to  assess  the  ability  of  the  automated  DLT 
and  PMT  tasks  to  predict  primary  flight  grades  aid  successful  completion  of 
primary  flight  training,  two  important  criteria  in  the  Navy  flight  training 
program. 

Subjects 

To  date,  98  student  naval  aviators  (SNAs)  who  performed  on  the  backward 
series  of  tasks  and  105  SNAs  who  k  urformed  on  the  forward  series  have 
co:<  plated  or  failed  primary  training.  All  subjects  were  volunteers  for  the 
exoeriment. 
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Apparatus  and  Procedure 

Testing  apparatus  was  identical  to  that  described  in  Study  I.  Subject 
testing  was  conducted  while  SNAs  were  awaiting  entry  into  the  primary 
flight  portion  of  the  Navy  flight  training  program.  In  addition  to  DLT  and 
PMT  performance,  Naval  and  Marine  Aviation  Selection  Battery  test  scores 
were  obtained  for  each  subject.  The  battery  consists  of  the  Aviation 
Qualification  Test  (AQT)  and  the  Flight  Aptitude  Rating  (FAR).  The  AQT  is 
a  general  ability  test  of  verbal  and  quantitative  aptitude.  The  FAR  is 
comprised  of  a  Spatial  Apperception  Test,  a  Mechanical  Comprehension  Test, 
and  a  Biographical  Inventory.  All  subjects  had  been  initially  selected  for 
flight  training  based  on  their  performance  on  these  tests.  Criterion 
performance  measures  were  completion  or  failure  in  primary  flight  training 
(pass/fail)  and  primary  flight  grades  (PFGs).  In  addition,  the  relation  of 
subjects'  previous  flight  hours  (PFH)  to  the  various  test  measures  was 
incorporated  into  the  data  analysis. 

Results 


The  correlations  in  Tables  7A  and  7B  indicate  the  relation  of  the 
automated  forward  and  backward  task  and  AQT/FAR  measures  to  criterion 
performance  (pass/fail,  PFG,  and  PFH).  The  correlations  between  the 
various  measures  and  pass/fail  are  point-biserial  (r-pbi)  correlations 
while  those  between  PFG  and  PFH  are  Pearson  product-moment  correlation 
coefficients. 

The  correlation  results  indicate  that  neither  the  forward  nor  the 
backward  series  of  automated  DLT  or  PMT  tests  was  significantly  correlated 
with  pass/fail.  (The  FAR  was  correlated  with  pass/ fail  in  the  group  that 
took  the  backward  series.)  Theoretically,  the  multitask  tests  should 
predict  flight  failure  attrition  since  this  attrite  category  is  reportedly 
associated  with  problems  in  controlling  the  aircraft  and  in  performing 
multiple  tasks  (9).  In  examining  the  types  of  attrition  occurring  for  the 
two  groups  of  subjects  (20  attrites,  10  per  group),  there  were  only  two 
flight  failures,  and  both  of  these  were  in  the  forward  test  series  subject 
group.  The  remaining  attrite  categories  were:  Not  Aviator  Material  (lack 
of  motivation  expressed),  12  subjects;  Not  Physically  Qualified,  2 
svbjects;  Academic,  2  subjects;  and  Other,  2  subjects.  We  would  not  expect 
the  multitask  tests  to  predict  the  majority  of  these  failures,  which  are 
associated  with  "drop  on  reques t/mo tiva tion"  in  the  research  literature 
(8,9).  It  remains  to  be  seen  whether  an  accumulation  of  flight  failure 
attrition  will  improve  the  prediction  of  pass/fail. 

No  test  measure  was  significantly  related  to  PFH  except  the  AQT  for 
the  forward  group  of  test  subjects.  Importantly,  both  the  forward  and 
backward  series  had  some  automated  task  measures  significantly  correlated 
with  PFG.  For  the  backward  series,  all  test  measures  were  significantly 
correlated  with  PFG.  The  PMT  single-task  stick-and-rudder  measure  produced 
tne  strongest  correlation  (r  ■  -.49,  p  <  .01)  with  PFG.  For  the  forward 
series,  four  of  the  nine  test  measures  were  significantly  correlated  with 
PFG  with  the  initial  multitask  DLT  measure  producing  the  strongest 
correlation  (£  =  .31,  £  <  .01). 

To  statistically  determine  which  measures  would  predict  PFG,  forward 
selection  multiple  regression  analyses  were  conducted.  The  regressions 
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u-iliaed  all  test  measures  of  Table  7A  and  7B.  For  the  forward  aeries  of 
DLT  and  PMT  tasks,  only  the  Initial  multitask  DLT  (measure  2)  contributed 
significant  variance  to  the  regression  (£  *  .31,  £  (1,93)  <■  10.11,  j> 

<  .002).  Mo  other  measure  entered  the  regression  due  to  high  correlations 
among  the  other  test  measures  significantly  related  to  PFG. 


TABLE  7A.  Relation  of  Forward  Series  of  Automated  Tasks  and 

Selection  Test  Measures  to  Pass/Fail,  Primary  Flight  Grade 
(PFG),  and  Previous  Flight  Hours  (PFH) . 


Test  measures 

Pass/ Fail 
(n«105) 
"r-pbi 

PFG 

(n-95) 

""j: 

PFH 

(£•105) 

_r 

1. 

Single- task  DLT 

.11 

.22* 

-.03 

2. 

Initial  multitask  DLT 

.06 

.31** 

.04 

3. 

Second  multitask  DLT 

.00 

.18 

.11 

4. 

Third  multitask  DLT 

.13 

.10 

.03 

5. 

Single  PMT  stick 

.06 

.07 

.07 

6. 

Initial  multitask  PMT 

-.05 

-.24* 

-.04 

7. 

PMT  stick  &  rudder 

.00 

-.11 

-.11 

8. 

Second  multitask  PMT 

-.07 

-.20* 

-.16 

9. 

Third  multitask  PMT 

.02 

-.17 

-.12 

10. 

AQT 

-.04 

.10 

-.20* 

11. 

FAR 

-.08 

.06 

.17 

TABLE  7B.  Relation  of  Backward  Series  of  Automated  Tasks  and 

Selection  Test  Measures  to  Pass/Fall,  Primary  Flight  Grade 
(PFG),  and  Previous  Flight  Hours  (PFH). 


Test  measures 

Pass/  Fail 
(n-98) 
r-pbl 

PFG 

(n»88) 

jr 

PFH 

(n-98) 

r 

1. 

Single- task  DLT 

-.02 

.23* 

-.07 

2. 

Initial  multitask  DLT 

-.03 

.26* 

-.01 

3. 

Second  multitask  DLT 

.02 

.27* 

-.02 

4. 

Third  multitask  DLT 

.02 

.31** 

.00 

5. 

Single  PMT  stick 

-.10 

-.33** 

-.12 

6. 

Initial  multitask  PMT 

.01 

-.28** 

-.06 

7. 

PMT  stick  &  rudder 

.06 

-.49** 

-.06 

8. 

Second  multitask  PMT 

,06 

-.29** 

-.08 

9. 

Third  multitask  PMT 

.07 

-.33** 

-.07 

10. 

AQT 

.05 

.12 

-.10 

11. 

FAR 

.23* 

.34** 

.02 

*p 

<  . 05  (two-tailed) 

**£ 

<  .01  (two-tailed) 

For 

the  backward  series,  the 

single- task 

PMT  stick-end 

-rudder  (measure 

7)  entered  the  regression  equation  first  (R  -  .49),  followed  by  the  FAR 
Selection  test  (R_  ■  .53).  Each  of  these  measures  contributed  significant 
variance  to  the  final  regression  equation,  (F  (2,85)  *  16.56,  £  <.0001). 
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A  final  regression  analysis  was  performed  for  the  backward  test  series 
in  which  the  selection  test  measures  (AQT,  FAR)  were  forced  into  the 
regression  in  first  and  second  place  resulting  in  a  multiple  R  of  .35.  The 
PMT  stick-and-rudder  measure  then  entered  the  equation  and  contributed  an 
additional  16%  significant  variance  above  that  provided  by  the  selection 
tests  (R  -  .53,  F  (3.84)  -  11.10,  £  <  .0001). 

CONCLUSIONS  AND  RECOMMENDATIONS 

Statistical  analysis  of  Study  I,  comparing  a  forward  and  backward 
series  of  automated  DLT  and  PMT  tasks,  indicated  that  a  backward 
orientation  associated  with  the  psychomotor  tests  resulted  in  increased 
difficulty  for  all  PMT  measures  and  two  of  three  multitask  DLT  measures. 

The  correlational  estimates  of  test-retest  reliability  for  the  multitask 
DLT  and  PMT  measures  were  adequate  for  both  series  of  automated  tasks  but 
slightly  higher  for  the  backward  series.  There  were  significant 
correlations  between  the  DLT  and  PMT  tasks  or  each  series  of  single  and 
multitask  measures.  The  relation  between  such  seemingly  different  tasks  is 
difficult  to  understand  since  the  DLT  is  an  auditory  cognitive  processing 
task,  and  the  PMT  is  an  eye,  hand,  foot  coordination  task.  The  significant 
correlations  were  both  smaller  and  fewer  in  number  for  the  backward  series 
of  automated  tests. 

Study  II  was  an  evaluation  of  the  correlational  relation  between  the 
new  automated  multitask  measures  and  old  nonautomated  tasks  with 
demonstrated  validity  for  the  prediction  of  primary  flight  performance  (3). 
The  correlations  between  corresponding  tasks  of  the  new  automated  and  old 
nonautomated  tasks  averaged  .60  for  the  DLT  measures  and  .66  for  the  PMT 
tasks. 


The  results  of  Study  HI  indicated  that  certain  of  the  automated  DLT 
and  PMT  measures  were  significantly  related  to  primary  flight  grades  (PFG) 
in  Navy  flight  traiuing.  For  the  backward  series  of  tasks,  all  DLT  and  PMT 
measures  were  significantly  correlated  with  PFG.  In  contrast,  only  two  DLT 
and  two  PMT  measures  of  the  forward  series  were  significantly  related  to 
PFG.  No  significant  correlations  were  found  between  the  automated  DLT  and 
PMT  tests  and  the  pass/fail  criterion.  The  absence  of  a  suitable  number  of 
flight  failure  attritions  was  a  possible  reason  for  this  result.  A 
regression  analysis  for  the  backward  series  of  test  measures  and  primary 
flight  traiuing  criteria  indicated  that  a  psychoraotor  stick-and-rudder 
measure  and  the  FAR  selection  test  were  significantly  related  to  PFG. 

There  were  no  significant  correlations  between  the  automated  DLT  and  PMT 
measures  and  prior  flight  hours. 

These  research  results  indicate  that  a  series  of  automated  DLT  and  PMT 
tasks,  which  require  less  administrative  support  and  provide  automatic 
scoring  of  performance,  are  suitable  replications  of  an  older  version  of 
nonautomated  tasks.  The  backward  series  of  automated  tasks,  which  was 
correlated  more  strongly  with  criterion  performance,  should  be  administered 
to  a  large  sample  of  student  naval  aviators  to  determine  if  the  tests  can 
account  for  additional  variance  in  predicting  flight  training  performance 
beyond  that  of  current  selection  tests. 
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